However, it is insufficient to evaluate the stochastic algorithm not to consider the generalization performance, which is roughly the gap between Eq.

We also introduce anew on-average stability measure to develop optimistic bounds in a low noise setting.

Representative problems include AUC maximization [14, 25, 42, 63, 66], metric learning [8, 31], ranking [1, 13] and learning with minimum error entropy loss functions [29]. For example, in supervised metric learning we wish to find a distance function between pairs of examples so that examples within the same class are relatively close while examples from different classes are far apartfromeachother.

Finally, we obtain a generalization bound for meta learning withdependent episodes whose dependencyrelation ischaracterized byagraph.

It also illustrates that ergodicity is an important component for obtaining time-uniform bounds - which might not be achieved for convex or non-convex losses unless additional noise is injected to the iterates.